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(1) Real parties in interest 

The real parties in interest in the above-referenced patent application are 
Sony Corporation, a Japanese corporation with offices in Tokyo, Japan, and 
Sony Electronics Inc., a Delaware corporation with offices in New Jersey. 
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(2) Related appeals and interferences 

To the present knowledge of Appellants' legal representative, there are 
currently no related appeals or interference proceedings in progress which will 
directly affect, or be directly affected by, or have a bearing on the Board's 
decision in the present Appeal. 
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(3) Status of Claims 

Claims 1, 3-16, 18-21, 23-36, and 38-47 stand rejected under 35 U.S.C. 
§ 102(b), and claims 2, 17, 22, and 37 stand rejected under 35 U.S.C. § 103(a). 
The rejections of claims 1, 3-16, 18-21, 23-36, and 38-47, and the rejections 
of claims 2, 17, 22, and 37 are being appealed. 
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(4) Status of Amendments 

On January 3, 2007, a Final Office Action in the present Application was 
mailed to Applicants' Representative. In response, on March 30, 2007, the 
Applicants filed a Notice of Appeal in the present Application. 
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(5) Summary of Claimed Subject Matter 

In accordance with one embodiment of the present invention, a system user 
utilizes an electronic device 110 to capture audio/ video data (AV data) 226 while 
simultaneously providing a verbal narration 714 that is recorded as part of the AV 
data 226. In certain embodiments, when a label manager 218 instructs the 
electronic device 1 10 to enter a label mode, a speech recognition engine 214 of the 
electronic device 110 responsively performs speech recognition procedures upon 
the recorded AV data 226 (including the verbal narration 714) to automatically 
generate corresponding text labels. 222. 

In certain embodiments, the label manager 218 may optionally instruct a 
post processor 718 to perform appropriate post-processing functions on the text 
labels 222. For example, the post processor 718 may perform a validation 
procedure using one or more confidence measures to eliminate invalid text strings 
that fail to satisfy certain pre-determined criteria. The text labels 222 are then 
stored in any appropriate manner. For example, the label manager 218 may store 
each of the text labels 222 at different subject matter locations in the AV data 226 
depending upon where the corresponding original narration 714 occurred. The 
text labels 222 may also be stored separately along with certain meta-information 
(such as video timecode) that identifies specific subject matter locations in the AV 
data 226 that correspond to respective text labels 222. 

In a label search mode, the label manager 218 coordinates label search 
procedures for the electronic device 110. In certain embodiments, the label 
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manager 218 generates a label-search graphical user interface (GUI) upon a 
display 134 of the electronic device 1 10 for enabling a system user to utilize the 
text labels 222 to thereby locate corresponding sections of the AV data 226. In 
certain embodiments, the label search GUI includes, but is not limited to, a list of 
text labels 222 along with corresponding respective thumbnail images of 
associated video locations in the AV data 226. 

A system user may then select a desired search label by using any 
appropriate mesins. After a search label has been selected by the system user, 
then the label manager 218 instructs the electronic device 1 10 to automatically 
locate and display a corresponding section from the AV data 226. For at least the 
foregoing reasons, the present invention effectively provides an improved system 
and method for automatically cataloguing data by utilizing speech recognition 
procedures. 

Independent claim 1 recites "an electronic device that captures . . . 
The foregoing subject matter is discussed in the Specification, for example, at 
page 14, lines 23-32. Claim 1 also recites "a speech recognition engine that 
automatically performs . . . The foregoing subject matter is discussed in the 
Specification, for example, at page 15, lines 4-23. Claim 1 further recites "a 
label manager that manages a label mode for . . . The foregoing subject 
matter is discussed in the Specification, for example, from page 15, line 23 
through page 16, line 24. 

Independent claim 21 recites "capturing audio/ video data corresponding 
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to a photographic target by utilizing an electronic device . . . The foregoing 
subject matter is discussed in the Specification, for example, at page 14, lines 
23-32. Claim 21 also recites "providing a speech recognition engine that 
automatically performs . . . The foregoing subject matter is discussed in the 
Specification, for example, at page 15, lines 4-23. Claim 21 further recites 

"managing a label mode . . . and "controlling a label search mode " The 

foregoing subject matter is discussed in the Specification, for example, from 
page 15, line 23 through page 16, line 24. 

Independent claim 41 recites "capturing audio/ video data corresponding 
to a photographic target by utilizing an electronic device . . . The foregoing 
subject matter is discussed in the Specification, for example, at page 14, lines 
23-32. Claim 41 also recites "providing a speech recognition engine that 
automatically performs . . . The foregoing subject matter is discussed in the 
Specification, for example, at page 15, lines 4-23. Claim 41 further recites 

"managing a label mode . . . and "controlling a label search mode " The 

foregoing subject matter is discussed in the Specification, for example, from 
page 15, line 23 through page 16, line 24. 

Independent claim 42 includes four elements that are recited utilizing 
"means plus function" language. Independent claim 42 recites "means for 
capturing . . . The foregoing subject matter is discussed in the Specification, 
for example, at page 14, lines 23-32. Claim 42 also recites "means for 
automatically performing . . . The foregoing subject matter is discussed in 
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the Specification, for example, at page 15, lines 4-23. Claim 42 further recites 
"means for managing . . . ," and "means for controlling . . . The foregoing 
subject matter is discussed in the Specification, for example, from page 15, line 
23 through page 16, line 24, 

Independent claim 43 recites "an imaging device that captures . . . The 
foregoing subject matter is discussed in the Specification, for example, at page 
14, lines 23-32. Claim 43 also recites "a speech recognition engine that 
automatically performs . . . The foregoing subject matter is discussed in the 
Specification, for example, at page 15, lines 4-23. Claim 43 further recites "a 
label manager that manages a label mode during which The foregoing 
subject matter is discussed in the Specification, for example, from page 15, line 
23 through page 16, line 24. . 

Independent claim 44 recites "an electronic device that captures . . . 
The foregoing subject matter is discussed in the Specification, for example, at 
page 14, lines 23-32. Claim 44 also recites "a speech recognition engine that 
automatically performs . . . The foregoing subject matter is discussed in the 
Specification, for example, at page 15, lines 4-23. Claim 44 further recites "a 
label manager that manages a label mode for . . . The foregoing subject 
matter is discussed in the Specification, for example, from page 15, line 23 
through page 16, line 24. 

Independent claim 45 recites "an electronic device that captures . . . 
The foregoing subject matter is discussed in the Specification, for example, at 
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page 14, lines 23-32. Claim 45 also recites "a speech recognition engine that 
automatically performs . . . The foregoing subject matter is discussed in the 
Specification, for example, at page 15, lines 4-23. 

Independent claim 46 recites "an electronic device that captures . . . 
The foregoing subject matter is discussed in the Specification, for example, at 
page 14, lines 23-32. Claim 46 further recites "a label manager that manages 
a label mode for . . . The foregoing subject matter is discussed in the 
Specification, for example, from page 15, line 23 through page 16, line 24. 

Independent claim 47 recites "capturing electronic data which includes a 
narration The foregoing subject matter is discussed in the Specification, 

for example, at page 14, lines 23-32. Claim 47 also recites "performing a 
speech recognition process The foregoing subject matter is discussed in 

the Specification, for example, at page 15, lines 4-23. Claim 47 further recites 
"utilizing said labels . . . The foregoing subject matter is discussed in the 
Specification, for example, from page 15, line 23 through page 16, line 24. 

The subject matter of dependent claims 4 and 24 is discussed in the 
Specification, for example, at page 17, lines 3-32. The subject matter of 
dependent claims 5, 14, 18, 25, 34, and 38 is discussed in the Specification, 
for example, at page 15, lines 9-12, and at page 16, lines 1 1-14. The subject 
matter of dependent claims 6 and 26 is discussed in the Specification, for 
exaimple, at page 17, lines 3-32. 

The subject matter of dependent claims 7, 12-13, 27, and 32-33 is 
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discussed in the Specification, for example, at page 15, lines 23-28. The 
subject matter of dependent claims 2 and 22 is discussed in the Specification, 
for example, at page 8, lines 1-4. The subject matter of dependent claims 17 
and 37 is discussed in the Specification, for example, at page 20, lines 10-23. 
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I. Rejections of claims 1, 3-16, 18-21, 23-36, and 38-47 under 35 U.S.C. § 
102(b) as being anticipated by U.S. Patent No. 5,649,060 to EUozy et al. 

II. Rejections of claims 2, 17, 22, and 37 under 35 U.S.C. § 103(a) as being 
unpatentable over U.S. Patent No 5,649,060 to EUozy et al. in view of U.S. 
Patent No. 6,807,367 to Durlach. 
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(7) Argument 

I. 35 U.S.C. S 102fb) 

On page 2 of the Office Action, the Examiner rejects claims 1, 3-16, 18- 
21, 23-36, and 38-47 under 35 U.S.C. § 102(b) as being anticipated by U.S. 
Patent No. 5,649,060 to EUozy et al. (hereafter Ellozv) . The Applicants 
respectfully traverse these rejections for at least the following reasons. 

"For a prior art reference to anticipate in terms of 35 U.S.C. §102, every 
element of the claimed invention must be identically shown in a single 
reference." Diversitech Corp. v. Century Steps, Inc., 7 USPQ2d 1315, 1317 
(CAFC 1988). The Applicants submit that Ellozv fails to identically teach every 
element of the claims, and therefore does not anticipate the present invention. 

(A). Claims 1. 3. 8-11. 15-16. 19-20. 21. 23. 28-31. 35-36. and 39-40 

Reglafding the Examiner's rejections of independent claims 1 and 21, 
Applicants submit claims 1 and 21 recite limitations are not taught or 
suggested either by the cited reference, or by the Examiner's citations thereto. 
For example, claim 1 recites "said audio/ video data including a narration 
provided by a narrator to identify respective subject matter locations in said 
audio/ video data" (emphasis added). 

Ellozv teaches a technique for aligning a pre-existing "written transcript" 
with ambient speech in an A/V recording. In particular, Ellozv teaches using a 
speech recognizer to convert the recorded ambient speech into decoded text 



that is associated with timestamps. EUozv then matches the written transcript 
to the decoded text to associate the timestamps with the written transcript as 
"index data" (see column 3, line 31 to column 4, line 67). Through this 
technique, EUozv converts the pre-existing written transcript into index data. 

In contrast, Applicants nowhere claim utilizing any type of pre-existing 
written transcript, as taught by EUozv . On the contrary, Applicants claim 
directly co nverting "narration" provided by a narrator (such as a device user of 
the A/V capture device) into the "labels" for performing search procedures. 
Furthermore, the narration is provided at the same time as the A/V data is 
recorded specifically " to identify respective subject matter locations in said 
audio/ video data" (emphasis added). Applicants submit that nowhere in the 
entire disclosure does EUoizv refer to or mention any type of "narration" or 
"narrator." 

On the contrary, EUozv teaches an extremely time-consuming and 
burdensome process of creating a written transcript and then comparing the 
written transcript to a text version of recorded audio information. Applicants 
submit that this is clear evidence that any recorded utterances in EUozv are 
initially unknown to the device user, and therefore no narrator is involved. 
Otherwise, why would EUozv engage is such a lengthy and tiresome procedure 
as the foregoing transcription and comparison processes? Applicants therefore 
submit that the written transcript of EUozv is merely ambient speech that was 
uttered by human subjects during the course of the A/V recording, and is not a 
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verbal "narration" produced by a "narrator," 

Applicants therefore respectfully submit that EUozy nowhere teaches "a 
narration provided by a narrator, " and more specifically fails to teach recording 
a device-user's narration that is provided expressly "to identify respective 
subject matter locations in said audio/ video dat<f (emphasis added), as claimed 
by Applicants. For at least the foregoing reasons, Applicants submit that 
claims 1 and 21 are not anticipated by the teachings of EUozy. 

Regarding the Examiner's rejection of dependent claims 3, 8-11, 15-16, 
19-20, 23, 28-31, 35-36, and 39-40, for at least the reasons that these claims are 
dependent from respective independent claims whose limitations are not 
identically taught or suggested, the limitations of these dependent claims, when 
viewed through or in combination with the limitations of the respective 
independent claims, are also not identically taught or suggested. Applicants 
therefore respectfully request reconsideration of dependent claims 3, 8-11, 15-16, 
19-20, 23, 28-31, 35-36, and 39-40. 

(B). Independent Claim 41 

Regarding the Examiner's rejections of independent claim 41, Applicants 
submit that claim 41 recites limitations are not taught or su^ested either by 
the cited reference, or by the Examiner's citations thereto. For example, claim 
41 recites "said audio/ video data including a narration provided by a narrator to 
identify respective subject matter locations in said audio/ video data" (emphasis 



added). 

Independent claim 41 recites certain limitations that are similar to 
corresponding limitations of independent claim 21 that were discussed in detail 
above. Applicants therefore here incorporate, with respect to independent 
claim 41, those arguments made above in connection with independent claim 
21. In addition, Applicants submit that independent claim 21 is directed 
towards a method for practicing Applicants' invention. However, independent 
claim 41 is directed towards a "computer-readable medium comprising program 

instructions " For at least the foregoing reasons, Applicants submit that 

claim 41 is not anticipated by the teachings of Ellozv . 

(C). Independent Claim 42 

With regard to claim 42, "means-plus-function" language is utilized to 
recite elements and functionality similar to those recited in claims 1 and 21, as 
discussed elsewhere. Applicants therefore incorporate those remarks by 
reference with regard to claim 42. In addition, the Courts have frequently held 
that "means-plus-function" language, such as that of claim 42, should be 
construed in light of the Specification. More specifically, means-plus-function 
claim elements should be construed to cover the corresponding structure, 
mxxterial or acts described in the specification, and equivalents thereof. In 
particular, independent claim 42 includes four elements that are recited 
utilizing "means plus function" language. Applicants respectfully submit that. 



in light of the substantial differences between the teachings of Ellozv and 
Applicants' invention as disclosed in the Specification, claim 42 is therefore not 
anticipated or made obvious by the teachings of Ellozv . 

P). Independent Claim 43 

Regarding the Examiner's rejections of independent claim 43, Applicants 
submit that claim 43 recites limitations are not taught or suggested either by 
the cited reference, or by the Examiner's citations thereto. For example, claim 
43 recites "said audio/ video data including a narration proiMed by a narrator to 
identify respective subject matter locations in said audio/ video data" (emphasis 
added). 

Independent claim 43 recites certain limitations that are similar to 
corresponding limitations of independent claim 1 that were discussed in detail 
above. Applicants therefore here incorporate, with respect to independent 
claim 43, those arguments made above in connection with independent claim 
1. In addition. Applicants submit that independent claim 43 recites a 
substantial number of additional limitations that are not present in 
independent claim 1. For example, claim 43 recites various specific details 
regarding a "label modeT and a separate "Zabe/ search mod^ that are both 
managed by a "label manager." For at least the foregoing reasons, Applicants 
submit that claim 43 is not anticipated by the teachings of Ellozv . 
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(E) . Independent Claims 44 and 47 

Regarding the Examiner's rejections of independent claims 44 and 47, 
Applicants submit that claims 44 and 47 recite limitations are not taught or 
suggested either by the cited reference, or by the Examiner's citations thereto. 
For example, claims 44 recites verbal narration data ' 'provided to identify 
respective subject matter locations in said audio/ video data'' (emphasis added). 
Independent claims 44 and 43 recite certain limitations (in a somewhat 
broader manner) that are similar to corresponding limitations of independent 
claims 1 and 21 that were discussed in detail above. Applicants therefore here 
incorporate, with respect to independent claims 44 and 47, those arguments 
made above in connection with independent claims 1 and 21. For at least the 
foregoing reasons, Applicants submit that claims 44 and 47 are not anticipated 
by the teachings of EUozy . 

(F) . Independent Claim 45 

Regarding the Examiner's rejections of independent claim 45, Applicants 
submit that claim 45 recites limitations are not taught or suggested either by 
the cited reference, or by the Examiner's citations thereto. For example, claims 
45 recites a narration provided by a narrator specifically ' 'to identifu respective 
subject matter locations in said audio/ video data'' (emphasis added). 
Independent claim 45 recites certain limitations that are similar to 
corresponding limitations of independent claim 1 that was discussed in detail 
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above. However, independent claim 45 fails to expressly recite a "label 
manager." Applicants therefore here incorporate, with respect to independent 
claim 45, those arguments made above in connection with independent claim 
1. For at least the foregoing reasons, Applicants submit that claim 45 is not 
anticipated by the teachings of EUozv . 

(G). Independent Claim 46 

Regarding the Examiner's rejections of independent claim 46, Applicants 
submit that claim 46 recites limitations are not taught or suggested either by 
the cited reference, or by the Examiner's citations thereto. For example, claims 
46 recites a narration provided by a narrator specifically "to identify respective 
subject matter locations in said audio/ video data" (emphasis added). 
Independent claim 46 recites certain limitations that are similar to 
corresponding limitations of independent claim 1 that was discussed in detail 
above. However, independent claim 46 fails to expressly recite a "speech 
recognition engine." Applicants therefore here incorporate, with respect to 
independent claim 46, those arguments made above in connection with 
independent claim 1. For at least the foregoing reasons. Applicants submit 
that claim 46 is not anticipated by the teachings of EUozv . 
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(H) . Dependent Claims 4 and 24 

Regarding the rejections of dependent claims 4 and 24, Applicants 
submit that Ellozv fails to teach a " real-time label mode for creating and storing 
said labels" while " concurrenthi capturing said audio/ video datcf (emphasis 
added), as claimed by Applicants. As discussed above, the technique of Ellozv 
is based upon initially generating a pre-existing "written transcript." Therefore, 
EUozy is unable to generate Applicants' claimed "labels" in real time while the 
audio/ video data is being captured. For at least the foregoing reasons, 
Applicants therefore submit that claims 4 and 24 are not unpatentable in light 
of the cited reference. Applicants therefore respectfully request withdrawal of 
the rejections of claims 4 and 24. 

(I) . Dependent Claims 5. 14. 18. 25. 34. and 38 

With regard to the rejections of dependent claims 5, 14, 18, 25, 34, and 
38, Applicants claim using the claimed "speech recognition engin^ to handle 
various system user commands for controlling corresponding claimed 
functionality of the present invention. Applicants submit that Ellozv fails to 
disclose or teach using a speech recognition engine to allow a system user to 
verbally c ontrol and interact with a video camcorder or other host system," as 
recited by Applicants in claims 5, 14, 18, 25, 34, and 38. Applicants therefore 
respectfully submit that the rejections of claims 5, 14, 18, 25, . 34, and 38 
under 35 U.S.C. §102 are improper. 
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(J). Dependent Claims 6 and 26 

Regarding the rejections of dependent claims 6 and 26, Applicants submit 
that EUozv fails to teach that "said speech recognition engine automatically 
generates said labels as said electronic device captures said audio/ video data and 
said narration'' (emphasis added), as claimed by Applicants. As discussed above, 
the technique of EUozv is based upon initially generating a pre-existing "written 
transcript." Therefore, Ellozv is unable to concurrentlv generate Applicants* 
claimed "labels" while the audio /video data is being captured . For at least the 
foregoing reasons. Applicants therefore submit that claims 6 and 26 are not 
unpatentable in light of the cited reference. Applicants therefore respectfully 
request withdrawal of the rejections of claims 6 and 26. 

(K). Dependent Claims 7. 12-13. 27, and 32-33 

With regard to the rejections of dependent claims 7, 12, 27, and 32, 
Applicants claim performing a post-processing procedure including a validation 
procedure using one or more confidence measures to eliminate invalid text labels 
that fail to satisfy pre-determined validation criteria" (emphasis added). 
Furthermore, in claims 13 and 33, Applicants recite "a label validation 
procedure for validating said text labels, said label manager generating a 
validation graphical user interface upon a display of said electronic device for a 
system user to interactively evaluate, delete, and edit said text labels'' (emphasis 



added). 

The Examiner cites column 7, lines 53-63, of Ellozv against the 
limitations recited in claims 7, 12, 27, and 32. Applicants respectfully 
traverse, and submit that the cited passage of Ellozv fails to disclose or teach a 
"validation procedure?' that uses "confidence measures to eliminate invalid text 
labels," as recited by Applicants in claims 7, 12, 27, and 32. In addition, 
Applicants further submit that Ellozv nowhere teaches using a "a validation 
graphical user interface" to allow a "system user to interactively evaluate, delete, 
and edit said text labels," as recited by Applicants in claims 13 and 33. 
Applicants therefore respectfully submit that the rejections of claims 7, 12, 13, 
27, 32, and 33, under 35 U.S.C. § 102(e) are improper. 

Because a rejection under 35 U.S.C. §102 requires that every claimed 
limitation be identically taught by a cited reference, and because the Examiner 
fails to cite Ellozv to identically teach or suggest the claimed invention, 
Applicants respectfully request reconsideration and allowance of claims 1, 3- 
16, 18-21, 23-36, and 38-47. 
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II. 35 U.S.C. S 103 

On page 8 of the Office Action, the Examiner rejects claims 2, 17, 32, 

and 37 under 35 U.S.C. § 103 as being unpatentable over Ellozv in view of U.S. 

Patent No. 6,807,367 to Durlach (hereafter Durlach). The Applicants 

respectfully traverse these rejections for at least the following reasons. 

Applicants maintain that the Examiner has failed to make a prima fade 

case of obviousness under 35 U.S.C. § 103(a) which requires that three basic 

criteria must be met, as set forth in M.P.E.P. §2142: 

"First, there must be some suggestion or motivation, either in the 
references themselves or in the knowledge generally available to one 
of ordinary skill in the art, to modify the reference or to combine 
reference teachings. Second, there must be a reasonable 
expectation of success. Finally, the prior art reference (or references 
when combined) must teach or suggest all the claim limitations " 
(emphasis added). 

The initial burden is therefore on the Examiner to establish a prima fade case of 
obviousness under 35 U.S.C. § 103(a). 

Applicants respectfully traverse the Examiner's assertion that 
modification of the device of Ellozv according to the teachings of Durlach would 
produce the claimed invention. Applicants submit that Ellozv in combination 
with Durlach fail to teach a substantial number of the claimed elements of the 
present invention. Furthermore, Applicants also submit that neither Ellozv nor 



Durlach contain teachings for combining the cited references to produce the 
Applicants' claimed invention. The Applicants therefore respectfully submit 
that the obviousness rejections under 35 U.S.C §103 are improper. 

(A). Dependent Claims 2 and 22 

With regard to claims 2 and 22, the Examiner concedes that EUozv fails 
to "explicitly teach wherein said electronic device is implemented as an 
audio/video camcorder device." Applicants concur. The Examiner then points 
to Durlach to purportedly remedy this deficiency in Ellozv . Applicants 
respectfully traverse. The Examiner cites column 1, lines 43-55, of Durlach to 
support the rejections of claims 2 and 22. Applicants submit that column 1, 
lines 43-55, of Durlach contains only the briefest mention of the word 
"camcorder," without any relevant teaching of a combination with Ellozv to 
produce Applicants' invention. 

The Examiner then cursorily concludes that "[tjherefore, it would have 
been obvious to one in the ordinary skill in the art . . . because . . . creating a 
new movie image content is relatively inexpensive and straightforward even for 
the individual consumer." Applicants respectfully submit that a general 
restatement of the advantages disclosed by the Applicants deriving from 
implementation of the present invention cannot act as the required teaching or 
suggestion to combine cited references for a proper rejection under 35 U.S.C. § 
103. Courts have repeatedly held that "it is impermissible . . . simply to engage 



in hindsight reconstruction of the claimed invention, using the Applicants' 
structure as a template and selecting elements from references to fill in the 
gaps.'' In re Gorman . 18 USPQ 1885, 1888 (CAFC 1991). 

With further regard to the rejections of claims 2 and 22, Applicants 
observe that, in general, both camcorders and speech recognition technology 
have been known for some time. However, Applicants submit that their unique 
solution of utilizing a "speech recognition engine" to efficiently create their 
claimed "labels" to facilitate a "label search mode" shows the clear existence of 
secondary indicia of non-obviousness . For example, there apparently has been 
a long-felt need for Applicants' solution in the relevant technological field. 
Furthermore, other entities and individuals in analogous arts have apparently 
failed to successfully overcome the foregoing problems in the manner disclosed 
by Applicants.. 

(B). Dependent Claims 17 and 37 

With regard to claims 17 and 37, the Examiner concedes that EUozv fails 
to "explicitly teach display." Apphcants concur. The Examiner then points to 
Durlach to purportedly remedy this deficiency in EUozv . Applicants 
respectfully traverse. The Examiner cites column 1, lines 12-17, of Durlach to 
support the rejections of claims 17 and 37. Applicants submit that column 1, 
lines 12-17, of Durlach nowhere teaches any sort of GUI. More specifically. 
Applicants submit that column 1, lines 12-17, of Durlach fails to teach "a label- 



search GUI on a display of said electronic device, a system user viewing said 
labels and corresponding representative images from said audio/ video data for 
selecting a search label, " as claimed by Applicants. 

For at least the foregoing reasons, the Applicants submit that claims 2, 
17, 32, and 37 are not unpatentable under 35 U.S.C. § 103 over EUozv in view 
of Durlach . and that the rejections under 35 U.S.C. § 103 are thus improper. 
The Applicants therefore respectfully request reconsideration and withdrawal of 
the rejections of claims 2, 17, 32, and 37 under 35 U.S.C. § 103. 
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SUMMARY 

For all the foregoing reasons, it is earnestly and respectfully requested 
that the Board of Patent Appeals and Interferences reverse the rejections of 
claims 1-47, so that the present Application may be allowed and pass to issue 
in a timely manner. 

Respectfully Submitted, 
Abrego et al. 



Date: 



By: 



Gregory J. Koerner, Esq. 
Registration No. 38,519 
Redwood Patent Law 
1291 East Hillsdale Boulevard, Suite 205 
Foster City, California 94404 
(650) 358-4000 
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(8) Claims Appendix 



1 . A system for cataloguing electronic information, comprising: 

an electronic device that captures audio/ video data corresponding to a 
photographic target, said audio/video data including a narration 
provided by a narrator to identify respective subject matter 
locations in said audio/ video data; 

a speech recognition engine that automatically performs a speech 

recognition process upon said narration to generate labels that 
correspond to said respective subject matter locations in said 
audio/video data; and 

a label manager that manages a label mode for generating and storing 

said labels, said label manager also controlling a label search mode 
for utilizing said labels to locate said respective subject matter 
locations in said audio/ video data. 

2. The system of claim 1 wherein said electronic device is implemented as an 
audio/video camcorder device. 

3. The system of claim 1 wherein said speech recognition engine is configured 
in a simplified configuration that efficiently compares said narration with acoustic 
models to identify phone strings that represent said narration, said speech 
recognition engine referencing a compact dictionary to look up recognized 
vocabulary words that correspond to said phone strings, said speech recognition 
engine utilizing a limited set of recognition grammar to form said recognized 
vocabulary words into said labels that are supported by said speech recognition 
engine. 
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4. The system of claim 1 wherein said label manager initially instructs said 
electronic device to enter a real-time label mode for creating and storing said 
labels, said electronic device concurrently capturing said audio/ video data and 
said narration after said label manager instructs said electronic device to enter 
said real-time label mode. 

5. The system of claim 1 wherein said electronic device enters a real-time label 
mode in response to a verbal label-mode command from a system user, said 
verbal label-mode command being recognized and provided to said label manager 
by said speech recognition engine. 

6. The system of claim 1 wherein said speech recognition engine automatically 
generates said labels as said electronic device captures said audio/ video data and 
said narration. 

7. The system of claim 1 wherein a post processor performs a post-processing 
procedure upon said labels in a real-time label mode, said post-processing 
procedure including a validation procedure using one or more confidence 
measures to eliminate invalid labels that fail to satisfy pre-determined validation 
criteria. 

8. The system of claim 1 wherein said label manager stores said labels 
during a real-time label mode, said labels being stored along with meta- 
information that associates each of said respective subject matter locations to a 
corresponding one of said labels. 

9. The system of claim 1 wherein said electronic device initially captures 
said audio/video data and said narration prior to entering said label mode. 



10. The system of claim 1 wherein said label manager instructs said electronic 
device to enter a non-real-time label mode for creating and storing said labels, 
said electronic device responsively retrieving and pla3dng back said audio /video 
data and said narration. 

1 1 . The system of claim 1 wherein said speech recognition engine automatically 
generates said labels by analyzing said audio/ video data and said narration as 
said electronic device plays back said audio/video data and said narration. 

12. The system of claim 1 wherein a post processor performs a post-processing 
procedure upon said labels in a non-real-time label mode, said post-processing 
procedure including a validation procedure using one or more confidence 
measures to eliminate invalid labels that fail to satisfy pre-determined validation 
criteria. 

13. The system of claim 1 wherein said label manager coordinates a label 
validation procedure for validating said labels, said label manager generating a 
validation graphical user interface upon a display of said electronic device for a 
system user to interactively evaluate, delete, and edit said labels. 

14. The system of claim 1 wherein said label manager coordinates a label 
validation procedure for validating said labels in response to verbal validation 
commands from a system user, said verbal validation commands being recognized 
and provided to said label manager by said speech recognition engine. 

15. The system of claim 1 wherein said label manager stores said labels in a 
non-real-time label mode, said labels being stored along with meta-information 
that associates each of said respective subject matter locations to a corresponding 
one of said labels. 
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16. The system of claim 1 wherein said label manager instructs said electronic 
device to enter said label search mode during which a system user interactively 
selects a search label for performing a label search procedure to locate a specific 
one of said respective subject matter locations corresponding to said search label. 

17. The system of claim 1 wherein said label manager generates a label-search 
GUI on a display of said electronic device, a system user viewing said labels and 
corresponding representative images from said audio/ video data for selecting a 
search label. 

18. The system of claim 1 wherein a system user selects a search label by 
issuing a verbal search-label command, said verbal search-label command being 
recognized and provided to said label manager by said speech recognition engine. 

19. The system of claim 1 wherein said label manager instructs said electronic 
device to automatically locate and retrieve a specific one of said respective subject 
matter locations in response to a system user selecting a search label. 

20. The system of claim 1 wherein said electronic device automatically plays 
back a specific retrieved one of said respective subject matter locations from said 
audio/ video data for viewing by said system user. 
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21. A method for cataloguing electronic information, comprising: 
capturing audio /video data corresponding to a photographic target by 

utilizing an electronic device, said audio/ video data including a 
narration provided by a narrator to identify respective subject 
matter locations in said audio /video data; 

providing a speech recognition engine that automatically performs a 
speech recognition process upon said narration to generate text 
labels that correspond to said respective subject matter locations 
in said audio /video data; 

managing a label mode for generating and storing said text labels by 
utilizing a label manager; and 

controlling a label search mode with said label manager, said label 
search mode utilizing said text labels to locate said respective 
subject matter locations in said audio/ video data. 

22. The method of claim 21 wherein said electronic device is implemented as an 
audio/ video camcorder device. 

23. The method of claim 21 wherein said speech recognition engine is 
configured in a simplified configuration that efficiently compares said narration 
with acoustic models to identify phone strings that represent said narration, said 
speech recognition engine referencing a compact dictionary to look up recognized 
vocabulary words that correspond to said phone strings, said speech recognition 
engine utilizing a limited set of recognition grammar to form said recognized 
vocabulary words into said text labels that are supported by said speech 
recognition engine. 
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24. The method of claim 21 wherein said label manager initially instructs said 
electronic device to enter a real-time label mode for creating and storing said text 
labels, said electronic device concurrently capturing said audio/ video data and 
said narration after said label manager instructs said electronic device to enter 
said real-time label mode. 

25. The method of claim 21 wherein said electronic device enters a real-time 
label mode in response to a verbal label-mode command from a system user, said 
verbal label-mode command being recognized and provided to said label manager 
by said speech recognition engine. 

26. The method of claim 21 wherein said speech recognition engine 
automatically generates said text labels as said electronic device captures S£ud 
audio/video data and said narration. 

27. The method of claim 21 wherein a post processor performs a post- 
processing procedure upon said text labels in a real-time label mode, said post- 
processing procedure including a validation procedure using one or more 
confidence measures to eliminate invalid text labels that fail to satisfy pre- 
determined Validation criteria. 

28. The method of claim 21 wherein said label manager stores said text 
labels during a real-time label mode, said text labels being stored along with 
meta-information that associates each of said respective subject matter 
locations to a corresponding one of said text labels. 

29. The method of claim 21 wherein said electronic device initially captures 
said audio /video data sind saiid narration prior to entering said label mode. 
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30. The method of claim 21 wherein said label manager instructs said electronic 
device to enter a non-real-time label mode for creating and storing said text labels, 
said electronic device responsively retrieving and playing back said audio/ video 
data and said narration. 

3 1 . The method of claim 2 1 wherein said speech recognition engine 
automatically generates said text labels by analyzing said audio /video data and 
said narration as said electronic device plays back said audio /video data and said 
narration. 

32. The method of claim 21 wherein a post processor performs a post- 
processing procedure upon said text labels in a non-real-time label mode, said 
post-processing procedure including a validation procedure using one or more 
confidence measures to eliminate invalid text labels that fail to satisfy pre- 
determined validation criteria. 

33. The method of claim 21 wherein said label manager coordinates a label 
validation procedure for validating said text labels, said label manager generating 
a validation graphical user interface upon a display of said electronic device for a 
system user to interactively evaluate, delete, and edit said text labels. 

34. The method of claim 21 wherein said label manager coordinates a label 
validation procedure for validating said text labels in response to verbal validation 
commands from a system user, said verbal validation commands being recognized 
and provided to said label manager by said speech recognition engine. 
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35. The method of claim 2 1 wherein said label manager stores said text labels in 
a non-real-time label mode, said text labels being stored along with meta- 
information that associates each of said respective subject matter locations to a 
corresponding one of said text labels. 

36. The method of claim 2 1 wherein said label manager instructs said electronic 
device to enter said label search mode during which a system user interactively 
selects a search label for performing a label search procedure to locate a specific 
one of said respective subject matter locations corresponding to said search label. 

37. The method of claim 2 1 wherein said label manager generates a label-search 
GUI on a display of said electronic device, a system user viewing said text labels 
and corresponding representative images from said audio/ video data for selecting 
a search label. 

38. The method of claim 2 1 wherein a system user selects a search label by 
issuing a verbal search-label command, said verbal search-label command being 
recognized and provided to said label manager by said speech recognition engine. 

39. The method of claim 21 wherein said label manager instructs said electronic 
device to automatically locate and retrieve a specific one of said respective subject 
matter locations in response to a system user selecting a search label. 

40. The method of claim 2 1 wherein said electronic device automatically plays 
back a specific retrieved one of said respective subject matter locations from said 
audio/video data for viewing by said system user. 
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41. A computer-readable medium comprising program instructions for 
cataloguing electronic information by: 

capturing audio/ video data corresponding to a photographic target by 
utilizing an electronic device, said audio/ video data including a 
narration provided by a narrator to identify respective subject 
matter locations in said audio/ video data; 

providing a speech recognition engine that automatically performs a 
speech recognition process upon said narration to generate text 
labels that correspond to said respective subject matter locations 
in said audio /video data; 

managing a label mode for generating and storing said text labels by 
utilizing a label manager; and 

controlling a label search mode with said label manager, said label 
search mode utilizing said text labels to locate said respective 
subject matter locations in said audio/ video data. 

42. A system for cataloguing electronic information, comprising: 

means for capturing audio /video data corresponding to a photographic 
target, said audio /video data including a narration provided by a 
narrator to identify respective subject matter locations in said 
audio/ video data; 

means for automatically performing a speech recognition process upon 
said narration to generate text labels that correspond to said 
respective subject matter locations in said audio/ video data; 

means for managing a label mode to generate and store said text labels; ' 
and 

means for controlling a label search mode that utilizes said text labels to 
locate said respective subject matter locations in said audio/ video 
data. 
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A system for cataloguing electronic information, comprising: 

an imaging device that captures audio / video data corresponding to 

selected photographic targets, said audio /video data including a 
verbal narration provided by a narrator to identify respective 
subject matter locations in said audio/ video data; 

a speech recognition engine that automatically performs a speech 

recognition process upon said narration to generate text labels that 
are based upon said narration, said text labels corresponding to 
said respective subject matter locations in said audio/ video data, 
said text labels including abbreviated word sequences that identify 
said selected photographic targets; and 

a label manager that manages a label mode during which said text labels 
are generated by said speech recognition engine, said label 
manager also storing said text labels during said label mode, said 
text labels being stored along with meta-information that 
associates said respective subject matter locations to 
corresponding ones of said text labels, said label manager also 
controlling a label search mode for utilizing said text labels to 
locate specific corresponding ones of said respective subject matter, 
locations from said audio /video data, said label manager providing 
a label-search user interface upon a display of said imaging device 
for displaying said text labels and corresponding visual images of 
said respective subject matter locations from said audio/ video 
data, a system user interactively choosing a selected text label by 
utilizing said label-search user interface, said imaging device 
responsively displaying said audio /video data from a selected 
subject matter location corresponding only to said selected text 
label. 
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44. A system for cataloguing electronic information, comprising: 

an electronic device that captures said electronic information that 

includes verbal narration data provided to identify respective 

subject matter locations in said audio/ video data; 
a speech recognition engine that analyzes said electronic information to 

generate labels that correspond to said respective subject matter 

locations in said electronic information; and 
a label manager that utilizes said labels to locate said respective subject 

matter locations in said electronic information. 



45. A system for cataloguing electronic information, comprising: 

an electronic device that captures audio/ video data corresponding to a 
photographic target, said audio /video data including a narration 
provided by a narrator to identify respective subject matter 
locations in said audio/ video data; and 

a speech recognition engine that automatically performs a speech 

recognition process upon said audio/ video data to generate labels 
that correspond to said respective subject matter locations in said 
audio /video data. 

46. A system for cataloguing electronic information, comprising: 

an electronic device that captures audio/ video data corresponding to a 
photographic target, said audio /video data including a narration 
provided by a narrator to identify respective subject matter 
locations in said audio /video data; and 

a label manager that controls a label search mode for utilizing labels 
derived from said narration to locate corresponding ones of said 
respective subject matter locations in said audio/ video data. 



47. An electronic cataloguing system implemented by: 

capturing electronic data which includes a narration provided by a 

narrator to identify respective subject matter locations in said 

audio /video data; 
performing a speech recognition process upon said electronic data to 

automatically generate labels that correspond to said respective 

subject matter locations in said electronic data; and 
utilizing said labels to locate said respective subject matter locations in 

said electronic data. 
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Evidence Appendix: None. 



(10) Related Proceedings Appendix: None. 



41 



